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Abstract 

Early comprehension is an important, but not well-understood, contribution to early literacy and language development. 
Specifically, research regarding the nature of skills representative of early comprehension, including how they contribute to 
later reading success, is needed to support best practices to adequately prepare students. This article describes the process 
involved in the creation and refinement of the newly developed comprehension Individual Growth and Development 
Indicators (IGDIs 2.0). Two theoretical models of early comprehension are discussed to highlight the inherent complexity 
of this domain. Results of three studies are presented: Study | outlines the initial piloting process, Study 2 represents a 
larger-scale investigation, and Study 3 describes further field testing and reveals the final |GDI 2.0 comprehension candidate: 
Which One Doesn’t Belong (WODB). Results indicated WWODB out-performed the other candidate measures across 
psychometric and pragmatic criteria. The utility of the WODB task within a Response to Intervention (RTI) framework is 


also discussed. 
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Discussions reflecting the transition from “learning to read” 
during the preschool years to a long-term goal of “reading 
to learn” are becoming increasingly prevalent in early child- 
hood (Dooley, 2011; Kail, 2011; Robinson, Einav, & Fox, 
2013; Storch & Whitehurst, 2002). It is during this transi- 
tion that children move from a focus on decoding text to a 
focus on deeper understanding, or comprehension, of the 
text and images printed on the page. Literacy experts agree 
that the end result of acquiring skills to read is comprehen- 
sion; however, there is much contention in the field about 
how and when the skills required to add meaning to words 
begin to develop and how the continuum of such compre- 
hension skills is manifested in the early years (van den 
Broek et al., 2005). 

At the same time, a renewed focus on conceptually and 
empirically validating models of comprehension from early 
childhood through high school completion has come to the 
forefront of research efforts with grant competitions such as 
the Reading for Understanding initiative hosted by the 
Institute of Education Sciences, and numerous projects 
recently funded to explore comprehension development and 
application such as Making the Right Connections: 
Improving the Comprehension of Struggling Readers 
(McMaster & van den Broek, 2011) and The Language 


Bases of Reading Comprehension (Justice, 2010). Yet few 
studies have demonstrated models that empirically and con- 
ceptually describe the nature of early comprehension. These 
initiatives illustrate the juvenile state of the field in discern- 
ing a robust continuum of pre-requisite and precursor skills 
representative of comprehension. 

Simultaneous to the K-12 efforts to explore comprehen- 
sion (e.g., Justice, 2010), a growing emphasis on the need to 
discern what comprehension is for preschool-age children 
has also been fostered as initiatives in the early childhood 
realm consider and explore the mechanisms contributing to 
achievement gaps in later elementary years (e.g., Tompkins, 
Guo, & Justice, 2013). Such efforts have led to re-examina- 
tion of current conceptualizations of how comprehension 
skills are acquired during early childhood. 

Investigations of early comprehension have been 
grounded in extrapolations from what we currently know and 
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understand about comprehension in the early elementary 
school years. Comprehension skills characteristic of later 
elementary reading abilities are supported by research dem- 
onstrating that there are necessary precursors and in some 
cases pre-requisite skills representative of learning to read 
(e.g., early literacy skills: alphabet knowledge, vocabulary, 
and phonological awareness; National Early Literacy Panel 
[NELP], 2008; Snow, Burns, & Griffin, 1998; Storch & 
Whitehurst, 2002; Whitehurst & Lonigan, 1998). Specifically, 
evidence suggests early vocabulary development predicts 
later comprehension (Dickinson & Smith, 1994; Sénéchal, 
Ouellette, & Rodney, 2006), and early phonological aware- 
ness and alphabet knowledge skills predict later decoding 
skills in the first and second grades (e.g., Bradley & Bryant, 
1983), with further studies demonstrating the connection 
between decoding and comprehension (Aarnoutse, Van 
Leeuwe, Voeten, & Oud, 2001). 

Early literacy skills represent at least four key domains 
(NELP, 2008; Sénéchal, LeFevre, Smith-Chant, & Colton, 
2001; Whitehurst & Lonigan, 1998) including (a) alphabet 
knowledge and concepts about print, or the ability to recog- 
nize and produce letter names and sounds and understand 
conventions of written text (McBride-Chang, 1999); (b) 
phonological awareness, or the ability to detect and manip- 
ulate words at the level of phonemes, the smallest units of 
spoken language (Anthony, Williams, McDonald, & 
Francis, 2007); (c) oral language, or a child’s expressive 
and receptive vocabulary and communication skills (Dunst, 
Trivette, Masiello, Roper, & Robyak, 2006); and (d) com- 
prehension, or the ability to gain information and draw 
inferences from written and/or spoken language (Snow 
et al., 1998). However, the continuum of how early compre- 
hension skills connect and contribute to later comprehen- 
sion is still unclear. A limited number of conceptual models 
are present in the research to demonstrate the development 
of early comprehension, with even fewer supported by 
empirical evidence. 


Theoretical Models of Early 
Comprehension 


Current research indicates there are factors that might con- 
tribute to the development of early comprehension, such as 
aspects of socio-cultural theory (Rojas-Drummond, 
Torreblanca, Pedraza, Velez, & Guzman, 2013) and social- 
cognitive learning theory (Glenberg & Gallese, 2012). Here, 
we review two current perspectives to aid in understanding 
the progression of young children’s early comprehension. 
Dooley and Matthews (2009) posit that early comprehen- 
sion is the cumulative sum of a tri-part processing system 
used to decode graphic symbols represented in words and 
images. The three parts of this system include the child (the 
early reader or decoder), the symbol present for the new con- 
cept (the actual text or picture), and the relationship between 


the child and the symbol (the activity in which the child 
encounters the new word or picture). Together, these compo- 
nents work in dynamic interaction to provide meaning for 
the child. As the child has additional experiences with the 
specific symbol within his or her socio-cultural context, 
meaning evolves, and the child develops a more complete 
understanding of the concept. For example, if a child’s first 
encounter with the word glasses is when a child sees a per- 
son in a book wearing glasses, his or her understanding may 
be limited to the context of their experience within this inter- 
action and will not yet include the meaning of glasses as the 
plural of drinking glass. However, a similar child might 
come from an environment where both parents and the child 
wear glasses, so encountering the word glasses would pres- 
ent a different level of understanding but may still be limited 
to only the concept of glasses as visual aids. Further, a third 
child who encounters the word g/asses and already has an 
understanding of glasses in the context of drinking vessels 
would add the new meaning of glasses (as visual aids) to his 
or her repertoire of information—supporting an evolved 
meaning to the word glasses. 

To support this tri-part system, Dooley and Matthews 
(2009) introduced three principles that set the stage for how 
comprehension may develop. First, they posited that young 
children develop comprehension in ways that are comple- 
mentary, but different from adults. This notion is also sup- 
ported by van den Broek and others in the field, indicating 
that preschool-age children are able to engage in the same 
inferential process and make meaningful connections, just 
as adults do (van den Broek, Kendeou, Lousberg, & Visser, 
2011). However, those inferences are limited relative to the 
child’s age. For example, preschool-age children are able to 
make relations between concrete, external, and individual 
events, but have a more difficult time comprehending rela- 
tions between abstract, internal, and clusters of events (van 
den Broek et al., 2011). 

Second, as events occur in children’s lives, they create a 
system of symbolic reference. Therefore, a child’s concepts 
of the words they encounter develop across time and experi- 
ence. As age and experience interact, children build a 
schema around each word they learn. As children age, these 
schemas become more homogeneous, conforming to the 
societal definition of each new word. In this way, symbols 
or referents for words evolve over time until a complete 
understanding is achieved (Dooley & Matthews, 2009). 

Third, comprehension is supported by socio-cultural fac- 
tors, such that children are motivated to learn new words 
based on the influence of primary caregivers and the early 
social environments in which they interact (Dooley & 
Matthews, 2009; Rojas-Drummond et al., 2013). Primary 
caregivers and peers in a young child’s life directly influence 
a child’s conceptions about new experiences. These influ- 
ences also guide the development of comprehension skills 
over time to provide a context for understanding. It is because 
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of these socio-cultural factors that children develop unique 
and specific vocabularies related to their environment. For 
example, a child who is raised in a home of musicians may 
have an extensive vocabulary including musical instruments, 
tempo and tone, just as a child who is raised in a home with a 
baseball aficionado might have an unusual depth and breadth 
of vocabulary around baseball players or pitches. 

Much like Dooley and Matthews (2009), van den Broek 
and colleagues contributed to conceptual arguments by fur- 
ther supporting that comprehension develops in preschool- 
age children in relationship to other knowledge but is 
experienced on a separate but complementary trajectory 
(2005). That is, van den Broek et al. reported comprehen- 
sion development occurs alongside early reading skills, not 
in succession to them, suggesting assessment and instruc- 
tional practices should attend to comprehension during the 
early years as a unique construct, rather than embedded in 
existing research regarding reading and knowledge 
acquisition. 

Foundations from Dooley and Mathews’s (2009) and 
van den Broek et al.’s (2005) model of comprehension 
suggest that early comprehension is a multi-dimensional 
phenomenon, created by building a family or collection 
of ideas, skill, and activities. For example, when stu- 
dents are asked, “What is a good teacher?” each could 
produce a different family of characteristics or skillset 
that may be necessary to their own definition. Students 
could identify a teacher as a “good teacher” for different 
reasons—because they are academically rigorous, com- 
passionate, patient, or yield a high percentage of high 
school graduates. In this way, children build different 
concepts of how they understand a word, question, or 
experience. 

Complementary to understanding comprehension as a 
collection of ideas, skills, or activities, is the more specific 
notion of making meaningful relations in text or narrative. 
Evidence suggests children use the context of the story, or 
the other sentences close in proximity to the target sentence 
to create relations. For example, if a teacher read two sen- 
tences: “The milk spilled on the table” and “The girl was all 
wet,” van den Broek et al. (2011) indicated that adults and 
children alike quickly create the relation that the girl’s wet 
clothing was attributed to the milk. This type of interaction 
happens in one of two ways, either an easy and nearly 
effortless process that happens automatically, as suggested 
with the process in the previous sentence, or in a slow, 
intentional, and strategic process that requires the attention 
and cognitive focus of the child that occurs when sentences 
have distal and indirect connections (van den Broek et al., 
2011). 

Taken together, Dooley and Mathews’s (2009) concep- 
tual presentation of early comprehension and van den 
Broek’s further contributions to early comprehension (van 
den Broek et al., 2005; van den Broek et al., 2011) suggest 


a foundation comprised of multi-dimensional concepts and 
meaningful relations, supplemented with three core process 
variables: type, efficiency, and content. First, van den Broek 
notes that adults and children engage in the same processes, 
but types of inferences are different as children age. For 
example, when completing story recall tasks, children and 
adults alike can recall similar static information, but the 
relations they establish between the referent information 
may be different. That is, inferences occur through early 
childhood in a continuum of complexity. Initially, children’s 
inferences are limited to concrete physical relations that 
occur close together in time. As children age, they move to 
concrete physical relations between distant events and then 
to casual relations involving character goals, emotions, and 
desires. Finally, children experience comprehension as hier- 
archical and thematic relations between clusters of events 
and to the most developed of inferences such as translation 
of a story theme into a moral or lesson (van den Broek et al., 
2011). 

Second, parallel to Dooley and Matthews’s (2009) 
model, van den Broek suggests that the core process vari- 
able of comprehension efficiency improves over time, such 
that experiences of drawing on a particular concept or word 
are reinforced or reduced over time. That is, experiences 
that align with correct interpretations of a word are rein- 
forced by others, while experiences that include misinter- 
pretations are extinguished. For example, if a child uses the 
word “long” to describe something in the context of its 
length, he will be reinforced of this use by others under- 
standing and agreeing with his statement (“Yes, that is a 
very long rope!”’). However if he then generalizes the word 
long to be used to illustrate height by saying, “David is very 
long!” others will extinguish the use of long by noting that 
a person is not long, instead the appropriate term is tall. The 
more experience a child has, the more complete the concept 
or schema. In this way, children build efficiency in develop- 
ing accurate representations of a word presented in narra- 
tive or written text. 

Finally, comprehension is also affected by the content 
delivered. Similar to Dooley and Matthews’s (2009) pre- 
sentation of socio-cultural influences, van de Broek pres- 
ents each opportunity for comprehension as a function of 
the information with which the child has had experience 
with. Evidence suggests children can and do gather large 
amounts of information with targeted interest groups (e.g., 
dinosaurs, chess, etc.; Chi & Koeske, 1983). For example, 
children interested in dinosaurs may independently form a 
rich understanding of dozens of different species and char- 
acteristics of dinosaurs, allowing for a deep capacity in 
dinosaur content. 

In sum, early comprehension is a valuable and meaning- 
ful component of early literacy that develops in conjunction 
with other skills (e.g., alphabet knowledge, phonological 
awareness etc.). However, the preliminary understanding of 
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the nature of comprehension described here is not enough to 
support effective instruction and intervention during the 
preschool years. We also need robust assessments to aid in 
understanding the progression of early comprehension 
skills with pre-kindergarten (Pre-K) students. In the follow- 
ing sections, we describe how such an assessment model 
may be pursued within the context of Response to 
Intervention. 


Response to Intervention 


To appropriately address the needs of all children’s compre- 
hension skills, assessment and intervention practices must 
be tailored to provide a match between a student’s skill 
level and instructional content (D. Fuchs, Fuchs, & 
Compton, 2012; C. R. Greenwood et al., 2011). The 
Response to Intervention (RTI) model is uniquely suited to 
address the varied needs of young learners by implementing 
a tiered system of assessment and intervention. 

RTI is a framework to identify, monitor, and intervene 
with students based on individualized student academic 
need (D. Fuchs & Fuchs, 2006; C. R. Greenwood, 
Kratochwill, & Clements, 2008). Students are assessed to 
determine level of current performance, and intervention 
services are provided to match this performance level in one 
of three tiers. Tier 1 features high-quality evidence-based 
instruction, with complementary periodic screening. Tier 2 
provides increased support for those students not making 
adequate progress in the general universal Tier | curriculum 
and is often presented as small group instruction along with 
more frequent progress monitoring to evaluate student per- 
formance. Tier 3 provides intensive, targeted, and individu- 
alized intervention and complementary progress monitoring 
for those students who continue to make limited progress 
with additional intervention. 

In an RTI model, measures used to assess early literacy 
skills must function in two ways (D. Fuchs & Fuchs, 2006; 
C. Greenwood, Carta, McConnell, Goldstein, & Kaminski, 
2008). First, data from measures must be able to be utilized 
to identify individual students who might require a more 
intensive level of intervention (Margolis, 2012). Second, 
for those students who are candidates for more intensive 
instruction and intervention, measures must accurately 
monitor progress over brief periods of time to continually 
evaluate if students are improving relevant skills during 
intervention (Margolis, 2012). Therefore, these measures 
must be psychometrically robust and logistically feasible, 
allowing educational professionals to gather meaningful 
data to inform instructional and intervention decisions. At 
the same time, assessments that demonstrate utility in an 
RTI model must also achieve additional empirical and prag- 
matic criteria. To demonstrate psychometric utility in 
assessing performance over brief periods of time, measures 
should obtain less than 20% of children with a score of zero, 


and produce skew and kurtosis values less than an absolute 
value of 1 (Wackerle-Hollman, Schmitt, Bradfield, 
Rodriguez, & McConnell, 2014). In addition, measures 
must also adhere to General Outcome Measurement 
(GOMs) tenets, including development of measures that are 
brief, (between 1 and 2 min per task), easy-to-administer, 
easy-to-interpret, related to long-term goals, and are inex- 
pensive (or easily attainable). GOM instruments also dem- 
onstrate longevity (can be used for at least one academic 
year), reliability, validity, and sensitivity to growth over 
time, with utility as a progress monitoring measure (L. D. 
Fuchs & Deno, 1991). 

Finally, pragmatic standards specific to RTI must also be 
achieved. First, measures must provide teachers and admin- 
istrators opportunities to examine data and make relevant 
and efficient instructional changes. Second, measures used 
in an RTI model may benefit from utilizing a criterion-refer- 
enced standard of performance, rather than comparison with 
a normative peer group, which limits the assessor’s ability to 
evaluate a student’s absolute skill level. Third, measures 
must include a psychometrically robust pool of items to rep- 
resent the construct of interest, rather than a small number of 
items which may not provide enough information about 
areas of skill deficit to inform instruction and intervention. 

Currently, few standardized assessments of comprehen- 
sion for preschool-age children exist, including, Renfrew Bus 
Story (Cowley & Glasgow, 1994), Clinical Evaluation of 
Language Fundamentals—2nd Edition (CELF Preschool-—2; 
Wiig, Secord, & Semel, 2004), and the Zest of Auditory 
Comprehension of Language—3rd Edition (Carrow- Woolfolk, 
1999). However, based on the measurement criteria previ- 
ously described, these types of standardized assessments do 
not meet the characteristics of RTI. In contrast, a wealth of 
GOMs exist for use in K—12 settings including the Dynamic 
Indicators of Basic Early Literacy Skills (DIBELS; Good & 
Kaminski, 2002) and Curriculum-Based Measures (Deno, 
1992) with complementary measures designed for Pre-K use, 
such as the Individual Growth and Development Indicators 
(IGDIs, McConnell, Priest, Davis, & McEvoy, 2002). IGDIs 
include two versions, the 1.0 version, designed for use in 
instructional decision making through fluency-based assess- 
ments of alliteration, rhyming, and picture naming, and the 
2.0 version specifically designed for use within an RTI 
framework (CEED@UROC, 2011). IGDIs 2.0 include five 
tasks: Picture Naming, Rhyming, Which One Doesn’t Belong 
(WODB), First Sounds, and Sound Identification. All IGDI 
2.0 tasks, and more specifically the comprehension measure, 
WODB, uses item response theory (Gorin, Embretson, & 
McKay, 2008; Wackerle-Hollman, Bradfield, McConnell, 
Albano, & Rodriguez, 2011) within Wilson’s (2005) mea- 
surement construction framework. Wilson’s framework was 
used to conceptually support item design, define item charac- 
teristics such as target responses, scoring rules, and to statisti- 
cally model item performance (for a detailed description of 
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Wilson’s model, see Wilson, 2005). Following Wilson’s sug- 
gestions, work on IGDIs 2.0 used Rasch modeling (Albano, 
Rodriguez, McConnell, Bradfield, & Wackerle-Hollman, 
2011). Rasch modeling provides an approach that considers 
both student ability (person parameter) and item-level statis- 
tics (item parameter). By locating items and student abilities 
on the same scale, we can better examine if items are avail- 
able for students at their given ability levels. Therefore, creat- 
ing items that surround and include the distribution of student 
ability offers the most parsimonious assessment of ability. 
This article presents the process used to develop, pilot, 
and validate the new comprehension IGDI 2.0 measures. 
These measures, utilizing the strengths of GOM and psy- 
chometric advances related to Wilson’s model and Rasch 
modeling, may provide a foundation for a robust measure- 
ment system for use in an early childhood RTI model. 
Comprehension, as previously described, is an area of 
early literacy that has proven difficult to capture develop- 
mentally, theoretically, and empirically (Dooley & 
Matthews, 2009; van den Broek et al., 2011); therefore, the 
measurement of comprehension skills was conducted over 
the course of three studies. Study | includes the primary 
development and initial measure design and selection, pilot 
psychometric and practical examination of several potential 
measures of comprehension. Study 2 includes further 
refinement and a selection of measures to undergo further 
testing. Study 3 includes the collection of evidence for 
validity and reliability with the refined set of measures. 


Study |: Developing and Piloting New 
Measures 


The purpose of Study | was to develop, pilot, and evaluate 
the comprehension IGDIs. Specifically, this study sought to 
answer the following research questions: 


Research Question 1: To what extent do the measures 
demonstrate characteristics of General Outcome 
Measurement (GOM)? 

Research Question 2: What are the demonstrated item- 
level characteristics? 

Research Question 3: What is the relation between the 
comprehension IGDI measures and also to other stan- 
dardized measures of early comprehension? 


To begin, we investigated the essential knowledge, skills, 
and abilities related to comprehension among preschool 
children. 


Defining Comprehension 


A literature review was completed to gain more insight into 
the comprehension construct. A keyword search for the 
term reading comprehension within the Psych Info database 


yielded 8,810 results. These were limited to 185 articles in 
peer-reviewed journals that were published in English and 
were focused on preschool-aged children. The titles and 
abstracts were reviewed to identify 33 articles that included 
an operational definition of one of the following terms: dis- 
course, listening, or narrative comprehension, text, passage, 
or story comprehension. These words and phrases were 
chosen to provide an inclusive lens of all research articles to 
date that offer a perspective on early comprehension. Target 
words and phrases were documented and supplied from 
seminal research articles including Dooley and Matthews’s 
(2009) and van den Broek’s work from 2005 to 2011. 

A review of the literature identified two skill areas repre- 
senting the construct during the preschool years: text com- 
prehension and_ listening comprehension. Text 
comprehension is defined as the ability to understand and 
interpret text, including pictures and symbols (Dunst et al., 
2006; Storch & Whitehurst, 2002). Listening comprehen- 
sion is the ability to understand and interpret what is spoken 
aloud and infer meaning from what is heard (Skarakis- 
Doyle, Dempsey, & Lee, 2008). 


Method 


Participants and setting. A total of 44 preschoolers enrolled 
in child care centers around a midwestern metropolitan area 
participated in this study. The child care centers selected 
represented a diverse sample of students based on socioeco- 
nomic status and ethnicity with 64% representing private- 
pay preschool environments, 23% representing urban public 
Pre-K programming, and 13% representing Head Start. Stu- 
dents recruited for this study were 68% Caucasian, 22% 
African American, 4% Asian American, and 6% other eth- 
nicities. Data collection occurred during the winter of 2010 
with children ranging in age from 39 to 63 months. Twelve 
of the children were 3-year-olds (36-47 months), 27 chil- 
dren were 4-year-olds (48-59 months), and 5 children were 
5-year-olds (60—71 months). In all, 55% were male. 


Measures. Four IGDI comprehension tasks and one stan- 
dardized criterion measure are described here. The IGDI 
comprehension measures were designed using an iterative 
process that initially included seven methods for assessing 
comprehension. Initial measure conception was driven by 
examining existing K-12 measures and a literature review 
summarization that yielded the extrapolation of current 
K-12 methods down to early childhood in adapted 
approaches and by developing novel approaches to the con- 
struct of comprehension through new task development. 
Initial measures included sentence comprehension, story 
comprehension, a story titles task, sequencing, picture com- 
prehension, a thematic inferences task and a categories task. 
Initial piloting reduced the pool to the three measures 
described here by eliminating those tasks that children did 
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not understand, that children did not respond to, or that 
were cumbersome to deliver. Once the three tasks were 
selected, an item-level analysis was completed to develop 
measures that included strategic foils (maximally or mini- 
mally different), reduced construct irrelevant variance, and 
were developmentally appropriate for Pre-K students. 


Picture Comprehension. The first IGDI, Picture Compre- 
hension, is an individually administered task in which the 
child is presented with an illustrated scene on a stimulus 
card. Following directions and administration of sample 
items, the assessor asked the child standardized “wh” ques- 
tions relevant to the item (e.g., What is happening in this 
picture? What is he doing? Where are they going? etc.). For 
example, when shown an item featuring an illustration of 
a girl wearing a bathing suit and carrying a beach towel, 
the examiner asked the child, “Where is she going?” The 
number of cards answered correctly in 2 min was recorded 
as the child’s score. 


Sequencing. Sequencing, the second IGDI tested, is an 
individually administered task in which the child is shown 
three related illustrations and is asked to place them in chron- 
ological order, given the first illustration in the sequence. 
Following instruction on how to complete the task and suc- 
cessful completion of two sample trials, the administrator 
presented the first image in the sequence (the target image) 
followed by two choices with the prompt: “This picture 
comes first” (points to the target image), “Which picture 
comes next?” (points to choice images). The assessor was 
provided with all correct responses on the back of each card 
as indicated by a star on the back of the correct image. The 
number of cards answered correctly in 2 min was recorded 
as the child’s score. 


Sentence Comprehension. Sentence Comprehension is an 
individually administered task in which the child is shown 
a card depicting three illustrations and is asked to choose 
(by pointing) the image that best matches the sentence that 
is presented. Following instruction on how to complete the 
task and successful completion of two sample trials, the 
administrator presented the card and described one of the 
three illustrations. For example, “The bird has flown away.” 
An item with three images is depicted: first a tree with a 
nest full of eggs and no bird; second, a bird flying toward 
the tree with the nest; and third, a bird sitting inside the nest. 
The assessor was provided with all correct responses on the 
back of each card as indicated by a star on the back of the 
correct image. The number of cards answered correctly in 2 
min was recorded as the child’s score. 


Story Comprehension. Story Comprehension is an indi- 
vidually administered task in which the child is asked to 
listen to a brief story available to general preschool popu- 


lations through public media resources (e.g., bookstores, 
online etc.) such as Leslie Patricelli’s The Birthday Box or 
Ruth Krauss’s The Carrot Seed. The administrator read the 
book aloud to the child and asked the child a series of 15 
brief questions about the story, including prediction ques- 
tions, direct recall questions, and inference questions. The 
child’s score was the number of correct responses. 


CELF Preschool-2. Three subtests of the CELF Pre- 
school-2 (Wiig et al., 2004) were administered to all chil- 
dren as a criterion measure of comprehension. The Core 
Language score is derived from the three subtests admin- 
istered. This standardized assessment is an individually 
administered test of language used to evaluate a child’s 
general language ability. Although the CELF Preschool-2 
is marketed as a language measure, it assesses facets of the 
comprehension domain, such as listening comprehension, 
making it an ideal criterion measure. For example, the Sen- 
tence Structure subtest evaluates the child’s ability to inter- 
pret spoken sentences of increasing length and complexity. 
The remaining two subtests (Word Structure and Expres- 
sive Vocabulary) evaluate the child’s ability to apply word 
structure rules and to label illustrations of people, objects, 
and actions. Test-retest reliability coefficients for 3- to 
5-year-olds range from .75 to .79 for Sentence Structure, 
.79 to .85 for Word Structure, and .78 to .95 for Expressive 
Vocabulary. The test-retest reliability coefficients for the 
Core Language score are high, ranging from .87 to .95 for 
3- to 5-year-olds (Wiig et al., 2004). 


Procedure. Measures were individually administered by 
undergraduate and graduate students who received compre- 
hensive training on all measures to ensure standardization 
and fidelity of implementation across the study. All asses- 
sors were monitored using fidelity checklists during train- 
ing, received feedback regarding administration errors, and 
were required to remedy errors before using the assess- 
ments with participating children. Initial fidelity statistics 
indicated assessors reached 81% fidelity on initial attempts 
and all assessors reached 100% fidelity by the second 
attempt during training. All assessment sessions were con- 
ducted on-site in a quiet hallway near the classroom. The 
assessments were conducted over the course of three ses- 
sions, each lasting from 15 to 20 min. After each session, 
the participants were allowed to select a small toy from a 
prize box. 

All measures were tested with a pool of 44 students with 
the exception of Story Comprehension, which included a 
reduced sample of 26 students. While the Story 
Comprehension was attempted with all 44 students, 18 stu- 
dents simply did not respond to the prompts indicating that 
they either were not comfortable with the administration 
setting or that they did not understand the prompt. These 
students did not respond to the assessor in any way. As a 
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Table |. Descriptive Statistics of Comprehension IGDIs Across Three Studies. 


Percentage 
Measure Wave N M SD Skewness Kurtosis zero scores 
Study | 
Picture Comprehension N/A 43 13.5 4.2 -0.9 1.7 | 
Sequencing N/A 43 10.4 3.2 0.4 0.5 0 
Sentence Comprehension N/A 42 9.8 5.0 0.1 0.0 2 
Story Comprehension N/A 26 5.0 2.1 -0.9 1.5 2 
Study 2 
Sentence Comprehension A N/A 88 9.1 2.2 -0.1 -0.5 12 
Sentence Comprehension B N/A 106 8.5 2.4 -0.2 -0.5 7 
Study 3 
Sentence Comprehension Winter 224 19.3 3.9 -0.6 -0.1 12 
WODB Winter 191 51.9 13.0 -1.0 -0.2 7 
Sentence Comprehension Spring 80 20.3 3.8 -0.6 -0.3 10 
WODB Spring 126 53.1 13.4 -1.1 -0.1 5 
Note. I|GDIs = individual growth and development indicators; WODB = Which One Doesn’t Belong. 
result, we did not include these occurrences as valid data or Table 2. Range of Item Means, Means, and Item-Total 
zero scores. Correlation Ranges by Measure Across Three Studies. 
Item-total 
Results Item means correlations 
Evaluation of measure criteria. Descriptive statistics for the % from % .20 
comprehension IGDI measures are presented in Table 1. aye ea 
ae — . : : Measure Range 80 Range above 
The descriptive statistics listed align with the suggested 
measurement criteria of GOMs (e.g., percentage zero Study | 
scores, skew, kurtosis, etc.). Results indicate the IGDI mea- Picture .17-1.00 36 —.35-.67 30 
sures met or exceeded the majority of psychometric criteria Comprehension 
for utilization in an RTI model previously described. Sequencing 28-91 80 “47-4440 
Sentence .19-1.00 80 -.13-.54 63 
Item-level performance. Item-level means and item-total CompEstiension 
correlations for each measure are displayed in Table 2. Item- ee? 
: : ; Sgt : Sentence 46-.96 77 -.02-.63 52 
level means _ provide information about individual item Comprehension A 
difficulty. Item-total correlations indicate the degree to ee 30-89 7\ 04-64 67 
which an item contributes to the overall measure and dis- Comprehension B 
criminates between those that do or do not have a trait (e.g., Study 3 
comprehension ability). For Sentence Comprehension and Sentence 30-98 63 07-42 70 
Sequencing, 80% of the item means fell between .20 and Composition 
.80. Conversely, only 36% of the item means for Picture WODB .26-.90 56 -.13--70 9 97 


Comprehension fell within the desired range. Sentence 
Comprehension had the highest percentage of items with 
item-total correlations greater than .20, while Picture Com- 
prehension again had the fewest. 


Relation between measures. Intercorrelation coefficients 
for the comprehension IGDI measures are provided in 
Table 3. Correlations ranged from low (7 = .28) to moderate 
(r = .53). The correlations between the IGDI comprehen- 
sion measures and the CELF Preschool—2 were moderate 
(r = .41-.74), with the exception of Sequencing, yielding a 
coefficient of r= .16. 


Note. WODB = Which One Doesn't Belong. 


Discussion 


The purpose of this pilot study was to examine the basic 
psychometric properties of the IGDI comprehension mea- 
sures and to determine which measures best demonstrated 
the characteristics of GOMs. Evaluation of each measure 
included a comparison of descriptive statistics to predefined 
GOM criteria, examination of correlations both within 
IGDI comprehension measures and with standardized crite- 
rion measures (e.g., CELF Preschool-2) to evaluate 
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Table 3. Intercorrelation Coefficients for Study | and Study 3. 


Picture Sentence Story Which One 
Measure Comprehension Sequencing Comprehension Comprehension Doesn’t Belong 
Picture | 
Comprehension 
Sequencing .28 | 
Sentence Av 37* | 
Comprehension 
Story Comprehension 53* AT* -40* | 
Which One Doesn’t N/A N/A 54* N/A I 
Belong 
CELF Sentence 43* 44* .63** 46* 74 
Structure 
CELF Word Structure 54* 16 55 .60** 57 
CELF Expressive .66* Al* 58% .66** 69% 
Vocabulary 


Note. Which One Doesn’t Belong was the only measure added in Study 3 and therefore results are presented collapsed across Study | and Study 3. 
Which One Doesn’t Belong correlations represent data reported in Study 3, whereas all other correlation values represent data reported in Study |. 


CELF = Clinical Evaluation of Language Fundamentals. 
*p < 05. **p < Ol. 


validity. Finally, item-level performance data were also 
examined to provide additional support for selecting mea- 
sures for further development and testing in Study 2. 


Essential GOM criteria. There are a number of essential cri- 
teria to consider when evaluating the extent to which a mea- 
sure demonstrates the characteristics of a GOM. As 
described previously, a GOM is quick and easy to use and 
interpret. The four comprehension measures evaluated in 
this study are brief in their administration (2 min each) and 
are easy to use. Another defining characteristic of a GOM is 
that it is psychometrically sound. Specifically, a psycho- 
metrically sound measure has a skew and kurtosis value that 
is less than the absolute value of one, and has a low number 
of zero scores. Results suggested that all the measures had 
reasonable skew and kurtosis and a low number of zero 
scores. The Story Comprehension measure yielded limited 
variability in total scores as a result of a constrained sample 
(n = 26), therefore inferences about utility are restricted. 


Validity evidence. To examine the relation between the mea- 
sures, intercorrelation coefficients were calculated. The 
correlations ranged from low (Sequencing and Picture 
Comprehension) to moderate (Story Comprehension and 
Picture Comprehension). The variability in the Intercorrela- 
tion coefficients suggests that the measures are not captur- 
ing the comprehension domain in the same way. 
Criterion-related validity evidence was examined by cal- 
culating correlations between each comprehension IGDI 
and the core subtests of the CELF Preschool—2. With the 
exception of Sequencing, all of the measures demonstrated 
moderate correlations with the CELF Preschool-2. 


Item-level functioning. Item-level information was not calcu- 
lated for Story Comprehension due to the restricted amount 
of information obtained from that measure. Item means, or 
p values, and item-total correlations were examined to 
understand how each measure functioned at the item level. 
The item mean indicates the specific difficulty of the item. 
The acceptable range of item means was within .20 and .80. 
Any item means outside this range did not meaningfully 
contribute to the assessment because they were either too 
difficult or too easy. Picture Comprehension had the lowest 
percentage of items that fell within the desired range (36%), 
whereas many were too easy. 

Item-total correlations are another criterion that helps 
determine how an item contributes to an assessment. 
Specifically, item-total correlations indicate the extent to 
which the items discriminate between children with high 
ability in the construct and children with low ability in the 
construct. Items with an item-total correlation of .20 or 
higher were determined to be contributing to the assess- 
ment. Sentence Comprehension had the highest percentage 
of items (63%) with item-total correlations greater than .20, 
suggesting that this measure had more items that were able 
to successfully discriminate between children’s ability lev- 
els. Picture Comprehension again had the lowest percent- 
age of items that met the desired criteria (30%). 

Considering all the relevant data, it was determined that 
Sentence Comprehension would move on to Study 2 for further 
testing and refinement. This measure demonstrated the most 
consistent combination of GOM characteristics, validity evi- 
dence and item-level functioning. It was also decided to com- 
bine Picture Comprehension and Sequencing into one task. 
Item-level data were used to make the decision to combine the 
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two tasks because neither demonstrated the characteristics of an 
effective measure on its own. The best items from each task 
were therefore combined to create one measure. 


Study 2 


Building on the results of the first study, Study 2 sought to 
answer similar research questions with a larger sample of 
children. Study 2 involved the following research 
questions: 


Research Question I: What are the basic psychometric 
properties of the IGDI comprehension measures? 
Research Question 2: What are the demonstrated item- 
level characteristics? 

Research Question 3: What is the relation between the 
comprehension IGDI measures and _ standardized 
measures of early comprehension? 


With Study 2 came a shift in the measurement frame- 
work from Classical Test Theory to the Rasch Model. The 
Rasch Model places items on a scale based on item diffi- 
culty defined as the ability required to have a 50% probabil- 
ity of correctly responding to the item, locating the average 
item at 0 (typically resulting in an ability scale from —4 to 
+4). Based on their performance on the IGDI items, chil- 
dren are assigned Rasch scores, which reflect their ability in 
the given domain, relative to the location of the items. Thus, 
items and children are placed on a common scale, defined 
by the items as representation of the construct. 


Method 


Participants and setting. Participants for Study 2 were 
recruited from early child care centers around a midwest- 
ern metropolitan area. Participants were 46% Caucasian, 
37% African American, 7% Hispanic, 5% Native Ameri- 
can, 4% Asian American, and 1% other ethnicities. Study 2 
occurred during the summer of 2010 and included 196 par- 
ticipants ranging in age from 36 to 69 months. Seventy of 
the children were 3-year-olds (36-47 months), 93 children 
were 4-year-olds (48-59 months), and 32 children were 
5-year-olds (60-69 months). There were limited data avail- 
able on the gender distribution of the sample; for this rea- 
son, gender information is described for 91 of the 196 
participants. Of the 91 participants, 61% were female and 
39% were male. 


Measures. The comprehension IGDIs administered in Study 
2 included Sentence Comprehension and the combined Pic- 
ture Comprehension and Sequencing (PCS) measures. 
Before being tested in Study 2, item-level revisions were 
made to each measure. Specifically, poorly functioning 
items were discarded or edited. 


For Sentence Comprehension, 27 items were tested. 
Three items were discarded from the previous study because 
of poor item-level functioning, and 19 were edited. Edits to 
the items included both changes to the prompt and changes 
to the images. Eight items received prompt changes. As an 
example, a card from Study | originally read, “The dog is 
barking at the cat.” For Study 2, the prompt was changed to 
read, “The dog is chasing the cat.” The change was made 
because it was too difficult to capture the original sentence 
with an image. A dog that is chasing a cat is better depicted 
through an image than a dog barking at a cat. 

Items also received image changes. Figure 1 shows an 
example of the kind of image changes made to a given item. 
The prompt for this particular item read, “Dad works.” The 
first set of images (top row) represents how the item looked 
in Study 1, whereas the second set of images (bottom row) 
represents the item in Study 2. The target image depicting 
dad working, the middle image, was changed to make the 
room look more like an office. The other two images, the 
distractor images, were altered so that they were maximally 
different from the target image. 

For PCS, a total of 38 items were tested. The best func- 
tioning items from each measure were combined (17 from 
Picture Comprehension, 21 from Sequencing). The admin- 
istration instructions were also altered to reflect both 
measures. 


Procedure. Study 2 participants were part of a larger study 
where each student was administered nine IGDI 2.0 mea- 
sures across four domains (Phonological Awareness, Oral 
Language, Alphabet Knowledge, and Comprehension) dur- 
ing the summer of 2010. Other tasks administered included 
three oral language tasks: Picture Naming, Definitional 
Vocabulary, and WODB; two phonological awareness 
tasks: First Sounds and Rhyming; and two alphabet knowl- 
edge tasks: Sound Identification and Letter Identification. 
Only the IGDI early comprehension measures are reported 
here, which included two tasks: Sentence Comprehension 
and PCS. Measures were individually administered by 
undergraduate and/or graduate students who received com- 
prehensive training on all measures to ensure standardiza- 
tion and fidelity of implementation across the study. All 
administration procedures and procedures for assessor 
fidelity of implementation were engaged in the same pro- 
cess described in Study 1. Initial fidelity statistics indicated 
assessors reached 84% fidelity on initial attempts and 100% 
fidelity by the second attempt during training. To decrease 
the burden on children (given each task had between 20 and 
30 items), a bundling procedure was created. The bundles 
were constructed such that each item was administered to at 
least 100 children. This was critical to collect sufficient 
item-level data to meet Rasch model requirements. Each 
bundle had four sample cards and five common cards that 
remained constant across bundles. 
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Figure I. Schematic of how images from an item in Study | (top row) changed in Study 2 (bottom row). 


Note. The prompt for the item read, “Dad works.” 


Results 


Characteristics of measures. Results of the PCS measure are 
not presented because administration of the measure dis- 
continued midway during the data collection period. The 
measure did not perform as expected and did not align with 
the characteristics of GOMs in terms of brevity and ease of 
use. 

Table 1 provides the descriptive statistics for Sentence 
Comprehension. Results are presented by bundle. Results 
suggested Sentence Comprehension yielded adequate 
mean, standard deviation (SD), skew, kurtosis, and percent- 
age of zero scores. 


Item-level performance. Item-level statistics for Sentence 
Comprehension are provided in Table 2. To reiterate, the 
item means provide information about individual item dif- 
ficulty. Item means ranged from r = .30 to .96. 


Relations between measures. To address the second research 
question, correlations with the CELF Preschool-2 were cal- 
culated. Table 4 demonstrates moderate correlations for 
each bundle of Sentence Comprehension with each subtest 


Table 4. Correlations With CELF Preschool-2. 


CELF CELF CELF 
Sentence Word Expressive 
Measure Structure Structure Vocabulary 
Sentence 56 36% 37 
Comprehension A 
Sentence 58 AP 56 


Comprehension B 


Note. CELF = Clinical Evaluation of Language Fundamentals. 
py < 01. 


of the CELF Preschool-2. Correlations yielded values of 
r= .36 to .58. 


Discussion 


Study 2 represented a larger-scale investigation into the util- 
ity and validity of the IGDI comprehension measures. The 
purpose of the study was to examine the basic psychometric 
properties of the IGDI comprehension measures and to 
ensure that they align with the characteristics of GOMs. To 
accomplish this, descriptive statistics and criterion-related 
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validity coefficients between the measures and the CELF 
Preschool—2 were examined. Item-level performance data 
were also examined to provide additional support for further 
development. 


Essential GOM criteria. Adopting the same evaluation crite- 
ria used in Study 1, each set or bundle of Sentence Compre- 
hension demonstrated reasonable ease of use, brevity, 
percentage of zero scores, skew and kurtosis values. This 
piece of evidence increases our confidence in the measure 
and indicates that Sentence Comprehension meets the crite- 
ria of aGOM. 


Validity evidence. Criterion correlations with the CELF Pre- 
school—2 were conducted as a source of criterion-related 
validity evidence. Each bundle of Sentence Comprehension 
demonstrated moderate correlations with the CELF' Pre- 
school-2 suggesting that Sentence Comprehension contin- 
ues to be an adequate indicator for measuring early 
comprehension skills. 


Item-level functioning. Item means were examined to get a 
better sense of how each measure functioned at the item 
level. The item mean indicates the specific difficulty of the 
item. The acceptable range of item means was within.20 
and .80. Any item falling outside this range is not contribut- 
ing in any meaningful way to the overall measure. More 
than 70% of items in each Sentence Comprehension bundle 
fell within the desired range. 

Using the information collected from the previous stud- 
ies, measures were selected for Study 3 based on a number 
of important factors. First, the extent to which the measures 
exhibited strong validity evidence with an established crite- 
rion measure within the comprehension domain was con- 
sidered. A second factor that was considered was the degree 
to which the measures aligned with the GOM characteris- 
tics. Third, each measure’s item-level information was con- 
sidered before deciding which measures moved forward in 
the study. As stated previously, the PCS measure failed mid- 
way during data collection and therefore was not selected 
for further testing. It was determined that Sentence 
Comprehension remained the best candidate for the com- 
prehension domain and required further testing and 
refinement. 


Study 3 


The purpose of Study 3 was to further examine the feasibil- 
ity and utility of Sentence Comprehension and to begin 
evaluating the WODB task as a measure of early compre- 
hension. The WODB task, previously developed as an oral 
language measure, transitioned to the comprehension 
domain after criterion correlations demonstrated higher 
relations between this measure and the CELF Preschool-2. 


In Study 2, WODB was used within the Oral Language 
domain (as reported in Bradfield, Besner, Wackerle- 
Hollman, Rodriguez, & McConnell, 2014). Given the lack 
of transparent transition from oral language to comprehen- 
sion, the WODB task was originally conceptualized as an 
oral language task because of its requirement to know and 
use language to make categories. However, after review of 
Study 2 performance, it was reasoned that WODB perfor- 
mance more accurately represented a child’s level of cate- 
gories or schemas for grouping and developing relations 
between words and concepts. Therefore, the WODB task 
was moved to the comprehension domain. Study 3 sought 
to answer the following research questions: 


Research Question I: What are the demonstrated item- 
level characteristics? 

Research Question 2: What is the relation between the 
comprehension IGDI measures and also to other stan- 
dardized measures of early comprehension? 


Participants and Setting 


A total of 599 children participated in Study 3. Data collec- 
tion occurred over two time periods, winter (March) and 
spring (May) of 2011. Participants were selected from local 
and distant preschool sites across the United States in eight 
states: Alaska, Michigan, Minnesota, Kansas, Missouri, 
Ohio, Indiana, and Oregon. Demographic data regarding 
ethnicity and SES were not available at the student level for 
this study because participating site teachers administered 
the IGDI tasks as part of their educational assessment pack- 
age. However, all sites were included in school districts that 
reported aggregated data as part of state reporting require- 
ments (information on each district’s ethnicity and SES 
may be requested from the author). Data to calculate age 
were not available for all children in the study. From the 
data that were available, the children’s ages ranged from 36 
to 69 months. Thirty-six of the children were 3-year-olds 
(36-47 months), 134 children were 4-year-olds (48-59 
months), and 154 children were 5-year-olds (60-69 
months). In all, 54% were male. 


Measures. Based on the conclusions from Study 2, Sentence 
Comprehension and WODB were selected for use to assess 
early comprehension in Study 3. A complete description of the 
Sentence Comprehension measure can be found in Study 1. 


WODB. WODB is an individually administered task in 
which the child is shown cards with three colored pictures. 
The child is asked to identify which of the three pictures 
does not belong with the others. Before the task begins, the 
assessor demonstrates how to correctly respond using four 
practice cards. The first two sample cards are modeled by 
the assessor; the child is asked to respond to the last two 
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sample cards with constructive feedback given for incor- 
rect responses. The child’s final score is the total number of 
questions answered correctly. 


Procedure. Study 3 participants were included as part of a 
larger validation study that included the administration of 
six IGDIs, representing the domains of early literacy (Pho- 
nological Awareness, Oral Language, Alphabet Knowledge, 
and Comprehension) during the winter and spring of 2011. 
The six tasks administered included the two comprehension 
measures, WODB and Sentence Comprehension; two pho- 
nological awareness tasks, Rhyming and First Sounds; one 
oral language task, Picture Naming; and one alphabet knowl- 
edge task, Sound Identification. Data collection for Study 3 
occurred over the course of two waves with the criterion 
measure administered during the winter data collection 
period. Assessment protocols mirrored those described in 
Studies | and 2; however, for all states besides Minnesota, 
classroom staff provided assessment interactions rather than 
trained graduate research associates (GRAs). In addition, all 
training for other sites was provided digitally using Adobe 
Connect software. Initial fidelity statistics indicated asses- 
sors reached 79% fidelity on initial attempts and 100% fidel- 
ity by the second attempt during training. 

For each comprehension measure, item-level revisions 
were made to improve image quality and to remove any con- 
struct irrelevant features that could inappropriately influence 
how children respond to the tasks. For example, Sentence 
Comprehension items changed from low resolution, black 
and white images in Study | to high resolution, color images 
in Study 3. Select WODB items were improved by removing 
unnecessary background features and to the extent possible, 
all illustrated images were replaced with photographic 
images. Administration instructions were not modified. 

During administration, each IGDI, excluding Sentence 
Comprehension, had roughly 60 items. There were a total of 
68 items for WODB and 27 items for Sentence Comprehension. 
To reduce the burden placed on children, each measure was 
administered across 3 days with a maximum of 20 items pre- 
sented in each interaction at each wave. In addition, for select 
sites, a matrix sampling technique system was instituted to fur- 
ther reduce child fatigue. As a result, any one student could 
have received measures in two domains, with the criterion 
measure sampled across the subsample ensuring that close to 
60 participants received the CELF Preschool-2. 


Results 


Characteristics of measures. Descriptive statistics for Sen- 
tence Comprehension and WODB are provided in Table 1. 
Results are presented by each wave of data collection. For 
both Sentence Comprehension and WODB, distributions 
suggest the measures do not indicate significant concern as 
illustrated by skew, kurtosis. However, the percentage of 


zero scores indicates a small percentage of students (5%— 
12%) were unable to respond to any of the WODB and Sen- 
tence Comprehension items. 


Item-level performance. Item-level means and item-total cor- 
relations for each measure are displayed in Table 2. For 
Sentence Comprehension, 63% of the item means fell 
between .20 and .80. For WODB, 56% of the item means 
fell within the desired range. WODB had the highest per- 
centage of items with item-total correlations greater than 
.20. 


Relation between measures. The correlation between the two 
comprehension IGDIs was moderate (r = .54). In addition, 
criterion correlations are presented in Table 3. Correlations 
with the IGDI measures and the CELF Preschool—2 were 
moderate, ranging from 7 = .55 to .74. 


Discussion 


This study presented a large-scale field test of the compre- 
hension IGDIs. The purpose of the study was to further 
examine the feasibility and utility of Sentence 
Comprehension and WODB as a measure of early compre- 
hension. Descriptive statistics, criterion correlations, and 
item-level statistics are offered as evidence of the measures’ 
utility. This study represented a diverse sample of students 
across five states in the United States. 


Essential GOM criteria. As described earlier in this manu- 
script, the hallmarks of a GOM are that the measure is brief, 
easy to score and administer, and psychometrically robust. 
Examining the descriptive statistics is one way of evaluating 
a measure’s alignment with GOM characteristics. The score 
distributions of each measure suggest that neither indicate 
significant concern as illustrated by skew, kurtosis. In gen- 
eral, Sentence Comprehension showed an improved distri- 
bution of scores over WODB. However, it is evident by the 
percentage of zero scores that some student abilities are not 
captured on the Sentence Comprehension or WODB tasks, 
indicating a floor effect for the lowest performers. In addi- 
tion, data collectors reported both tasks were easy to collect 
and interpret, brief to administer, and engaging for students. 


Item-level functioning. Item means and item-total correla- 
tions were examined to understand task functioning at the 
item level and how each item contributes to the overall 
measure. Whereas Sentence Comprehension had more item 
means that fell between the .20 and .80 range, WODB had 
the highest percentage of items with item-total correlations 
greater than .20. Although WODB had more items that were 
outside the item mean range, it had more items that were 
able to successfully discriminate between children’s ability 
levels (i.e., item-total correlations). 
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Validity evidence. To establish criterion-related validity for 
the comprehension measures, a subsample of students 
received the CELF Preschool-2. WODB yielded higher 
correlations with the comprehension criterion measure than 
did Sentence Comprehension, indicating that WODB may 
be a better measure of early comprehension skills. 

Based on the information presented here, as well as addi- 
tional considerations, including cost of development 
(Sentence Comprehension is significantly more expensive 
to develop because it requires an illustrator), potential for 
writing new items, and adequacy of the items in the current 
scale, WODB was suggested as the comprehension measure 
candidate for inclusion in the next cadre of IGDIs. 


Implications for Practice 


RTI in early childhood is gaining steam as a new and 
promising approach for addressing early academic chal- 
lenges, shifting the emphasis away from a “wait to fail” 
approach and toward a responsive and preventive approach 
to child intervention and assessment (C. R. Greenwood et 
al., 2011). The IGDIs 2.0 have been designed to be 
uniquely suited for use within an RTI model by including 
items that can reliably detect the ability levels of pre- 
school-age children and, thus, produce scores that can 
inform decisions about the need for intervention. As a 
result, IGDIs 2.0 demonstrate promise in an RTI frame- 
work and will be further developed through an iterative 
process—beginning with the methodology described 
within this article. 

More specifically, this manuscript delineates the itera- 
tive development process of the IGDI 2.0 comprehension 
tasks. The IGDI 2.0 comprehension tasks were developed to 
align with current theories of early comprehension. In par- 
ticular, the final IGDI 2.0 task, WODB, aligns with van den 
Broek’s (2005) position that early comprehension repre- 
sents a child’s ability to build a family or collection of 
knowledge. By asking children to draw on their existing 
knowledge of categories and collections, we can gather 
important information about their ability to comprehend 
information for use within the early literacy domain. 

However, even with the support of emerging theories of 
early comprehension it is important to note that the IGDI 
2.0 comprehension tasks still face significant challenges 
and require additional research and development. First, 
measures of early literacy that demonstrate utility in an 
early literacy RTI model must be able to capture the full 
continuum of skill development within a given construct. 
Specific to early comprehension, an ideal measure for an 
RTI model would have a limited, if any, floor effect. That is, 
the abilities of the lowest performers would be present on 
the task. IGDI 2.0 WODB, at present, has a non-negligible 
floor effect, suggesting room for improvement in task 
design. 


At the same time, the limited understanding of early 
comprehension available to inform task design provides a 
sort of catch-22. IGDI 2.0 comprehension work will con- 
tinue to explore new ideas about how to understand how 
children go through the process of making collections and 
families of knowledge that may represent lower levels of 
early comprehension abilities. 

It is also important to note that in the design process of 
IGDIs 2.0 comprehension tasks, we were aware of and 
attended to the relation between vocabulary and oral lan- 
guage development and comprehension. Whereas the theo- 
ries presented here suggest they are complementary 
constructs, they also are entangled through the reciprocal 
relationship present between knowledge and understanding. 
Further, we did not develop the IGDI 2.0 comprehension 
tasks to the exclusion of vocabulary and oral language. 
Instead, a complementary IGDI 2.0 task, Picture Naming, 
was developed to parallel the comprehension task (see 
Bradfield et al., 2014). 

Nevertheless, given the paucity of early GOM screening 
measures for comprehension, the work described within 
this manuscript provides evidence for the IGDI 2.0 compre- 
hension tasks suggesting potential for use within an early 
childhood RTI model. Results indicate the current task is 
able to accurately capture the ability levels of most pre- 
school-age students as demonstrated in Study 3 and may 
provide a resource for assessing the degree to which early 
comprehension skills have developed during the preschool 
years. 
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